lokvinfandomcom-20200215-history
Rcom tips
search xml output *http://us.qa3search.g3.reuters.com/rsearch/xmlOutput.do?q=apple&start=0&site=JP&sort=date:D:S:d1 check article contain video * article/2015/11/05/us-usa-fed-yellen-idUSKCN0ST22V20151105 * find article in NEWSGROUP table mysql> select PACKAGE_ID FROM NEWSGROUP WHERE UID='USKCN0ST22V20151105' \G; *************************** 1. row *************************** PACKAGE_ID: 940151762 * find in PACKAGENEWS table mysql> SELECT * FROM PACKAGENEWS WHERE ID=940151762 \G; *************************** 1. row *************************** ID: 940151762 DUID: tag:reuters.com,2015:newsml_KCN0ST22U:20 * use duid get xml from agency webservice ** http://10.90.39.223/mc-tools/mc.html ** tag:reuters.com,2015:newsml_KCN0ST22U:20 * find "video" like below 113294656 * the 113294656, and last part of tag:yospace.com,0000:rss_'113294656' , is yospace id * find in FEEDVIDEO table mysql> SELECT * FROM FEEDVIDEO WHERE FEED_STORY_ID=113294656 \G; admin tomcat cluster * admin tomcat config cluster info, meet a problem tomcat can't start. The cluster should config point to self ip clear apache disk cache * clear apache disk cache, you may find apache cache in folder like: /rt/apache/cache akamai staging env How to test using Akamai's Staging Network Applies To Description How do I test using Akamai's Staging Network? Akamai's Staging Network allows Akamai's customers to test configuration changes prior to going live on the Production Network. How exactly does one test from Staging? Testing involves two steps: first, find a Staging IP, and second, spoof your domain to this IP and test whether or not your site or application behaves as expected. 1) Here is how you find an Akamai Staging IP. Say, Akamai is serving your digital property www.foo.com. Typically, this digital property would be cnamed to an Edge Hostname such as www.foo.com.edgesuite.net or www.foo.com.edgekey.net. (Note, if this is a new property that has not been Akamaized yet, this cname may not be in place yet. In this case, you will have to create an Edge Hostname to which you will be cnaming prior to running this test.) $ dig www.foo.com ;; ANSWER SECTION: www.foo.com. 7200 IN CNAME www.foo.com.edgesuite.net. www.foo.com.edgesuite.net. 1359 IN CNAME a1234.g.akamai.net. To find a Staging IP, add "-staging" to your Edge Hostname as follows: www.foo.com.edgesuite-staging.net or www.foo.com.edgekey-staging.net Resolve the newly obtained hostname to find a Staging IP. $ dig www.foo.com.edgesuite-staging.net ;; ANSWER SECTION: www.foo.com.edgesuite-staging.net. 1264 IN CNAME a1234.g.akamai-staging.net. a1234.g.akamai-staging.net. 20 IN A 165.254.92.146 a1234.g.akamai-staging.net. 20 IN A 165.254.92.138 The "-staging" Edge Hostnames are created automatically when the Production Edge Hostnames "www.foo.com.edgesuite.net" or "www.foo.com.edgekey.net" are created. As with production Edge Hostname, the "-staging" Edge Hostnames resolve to optimal Staging IPs. 2) Once you have found a Staging IP, spoof www.foo.com to this IP and test your site or application as you would in Production. You can spoof by entering this domain and IP in your hosts file, or use testing tools such as curl, wfetch, or Akamai's homegrown tool EdgeSuite Booster available to download under Tools on the Luna Control Center. Testing on Staging is strongly recommended when you make your own changes to your configuration. It is required when a change is implemented by Akamai. You may also want to make sure that once you have correctly spoofed to the staging network, that your test requests are in fact going through it. KB Article 4856 explains these validation steps in further detail. If you are trying to perform functional or A/B testing for users in China on a property subscribed to China CDN Service, KB 4651 describes special considerations for that testing scenario. build rcom >sudo make switch app=rcom-betaus >sudo make context app=rcom-${edition} CATALINA_BASE=/rt/tomcat/cms rcom add new edition * http://10.90.22.6/reutersmediawiki/index.php/BjMediaDev/Staff/Liang_Zhang/Tips/CreateRcomModule * https://thehub.thomsonreuters.com/docs/DOC-904393 check article main text in newsdb * find uid in url http://www.reuters.com/article/2014/07/15/us-usa-immigration-idUSKBN0FK29N20140715 , the uid is "USKBN0FK29N20140715" * use uid search NEWSGROUP table mysql> SELECT * FROM NEWSGROUP WHERE UID='USKBN0FL07220140716' \G; UID: USKBN0FL07220140716 PACKAGE_ID: 833067408 ... MAIN_TEXT_ID: 935512347 MAIN_PICTURE_ID: 935512348 * use main_text_id search BASICPUBLICATION table mysql> SELECT * FROM BASICPUBLICATION WHERE ID="935512347" \G; ID: 935512347 FILE_NAME: tag:reuters.com,2014:newsml_KBN0FL072:1 REVISION_DATE: 2014-07-16 03:11:43 HEADLINE: Exclusive: U.N. monitors allege 'conspiracy' to divert Somali assets ... get dns info dnsinfo_ spotlight/install-cron.sh source /rt/baselayout/all.sh evalalltxtrecords echo $dnsinfo_g3mds_service_host activemq config in app box * find mdsloader-01 , /rt/properties less /rt/properties/quoteserver-static.properties #produsx-app-01 jms.queue.mdqueue-8.serverUrls=tcp://1 telent host port ^] >telnet quit find page from url * https://thehub.thomsonreuters.com/docs/DOC-831648 - cms page render workflow * find url /search, in cms db, site_map_url table select * from site_map_url where domain_id=7 and url='/search' \G; find no cache parameter * http://10.90.22.6/reutersmediawiki/index.php/Apache_caching_query_string_params web box /rt/apache/betaus/qs.txt * restart webbox apache will update qs.txt from db, and it's need cache server start cache-server$ sudo /etc/init.d/rcomcsd start qa env proxy info (ftp box) conf.d$ pwd /opt/apache22/conf/conf.d/proxy.conf proxy$ pwd /opt/apache22/conf/conf.d/proxy proxy$ grep -ir 23375 * rcom-vhosts.conf: rcom-vhosts.conf: ProxyRemote * http://10.90.108.116:23375 catch up http://10.90.1.7/cgi-bin/trac.cgi/wiki/NtransCatchup * on bdnloader box /rt/apps/ntrans-reader-server/server/catchup_script.sh shell>./catchup_script.sh MM/DD/YY channleId [ channleId2 ...] shell>./catchup_script.sh 11/17/14 40050030 telnet localhost 23012 LSREQ 1.0 SETMISSINGRANGE 40050030 1415836800000 1416212395000 ctrl + ] quit add slideshow to qa6 env * copy slide show tar file from qa2, on qa2-ftp-01 box /rt/shared/rmpd/reuters_rps/archives/ such as, /rt/shared/rmpd/reuters_rps/archives/2015-04-03/2015-04-03-qa.tar * copy this file to qa6-ftp-01 box, on folder /rt/shared/nas01/reuters_rps untar the file sudo tar xopf 2015-04-03-qa.tar * then the slideshow file will be dealwith by cron job shell, be sure ntrans will read this slideshow channel * find slideshow channle id by name SELECT * FROM news.CATEGORY where edition='us' and name='rcomus_all' video channel type * for slideshow there is only one channel in category table MrssFeedVideo SELECT * FROM news.CATEGORY where edition='US' and package_type='MrssFeedVideo' find related article * in PACKAGEMAP table find ROLE='Sidebar' package SELECT package_id, count(package_id) from PACKAGEMAP where role='sidebar' group by package_id order by package_id desc limit 1000; * find packageid which have more sidebar contents, use package_id find in PACKAGENEWS SELECT * FROM news.PACKAGENEWS where id = 894653316 article image location * image location , the "view" folder is large image ** the "thumb" is small image, but seldom use. /rt/shared/nas02/view/2015/03/30/106156436-1.JPG * the admin box has write permission to /rt/shared/nas02 , other box can't * the video image url like: http://10.35.60.120/resources/r/?d=20150330&i=106156436&w=161x90&r=106156436-1&t=2 the param , i=106156436 , or r=106156436-1 is image name the d=20150330 , is image folder clear qa activemq content * Activemq size too large, use entire /data02 disk clear sudo /etc/init.d/activemq stop sudo rm -rf /data02/rt/store/activemq/kahadb/* sudo /etc/init.d/activemq start check marketdata company description with upstream * com.reuters.rcom.dataloaders.marketdata.CompanyOverviewDataLoader * find repo number with ric (on mdsbulkmaster-01, marketdaba db) SELECT q.repno, q.ric, q.issue_PI, em.mdna_exchange, q.issuerid, q.companyname, q.country, q.primaryflag, q.type, q.category, q.tickersymbol, em.exchangename, eex.favored, q.issuer_PI FROM quoteview q JOIN exchangemap em ON em.mdna_exchange = q.exchange JOIN exchange_edition_xref eex ON eex.exchangeid = em.id JOIN edition ed ON eex.editionid = ed.id WHERE (q.ric = 'YUUZ.SI' and q.ric collate utf8_bin = 'YUUZ.SI') AND ed.id = 21 ORDER BY eex.favored DESC, eex.ordering ASC, q.primaryflag DESC; * another sql for find repono, may only use quoteview table; mysql> select * from quoteview where ric='yuuz.si'; * find companygeninfo mysql> select repno, productionDate, numEmployees, numEmployeesLastUpdated, totalSharesOut, commonShareholders, incorporatedState, incorporatedYear, publicSince, streetLine1, streetLine2, streetLine3, streetLine4, streetLine5, city, stateRegion, postalCode, country, contactPersonName, contactPersonTitle, email, mainPhone, mainFax, contactPhone, businessSummary, equityComposition, analystFootnotes, financialSummary from companygeninfo where repno = 'ABAA6' \G; * on mdsrt-01 find which job update table companygeninfo (rtmdb.bulk_job_name_value) appdev@produsx-mdsrt-01 mysql> select * from bulk_job_name_value where value='companygeninfo'; +----------------+----------------+ | name | value | +----------------+----------------+ | tables_geninfo | companygeninfo | +----------------+----------------+ * find the job on mdsbulkmaster-01 , geninfo$ pwd /rt/shared/nasmds/marketdata/sqlarchive/full/geninfo or /rt/shared/nasmds/marketdata/sqlarchive/incremental/geninfo grep -ir "ABAA6" * qa2 mysql vip can't access * when meet qa2 mysql vip can't access ~$ host 10.90.108.50 50.108.90.10.in-addr.arpa domain name pointer qa2-admindb-vip.g3.reuters.com * start vip shell ~$ sudo /etc/init.d/vip_QA2ADMIN_up stage apache rewrite * on staging box svn co http://10.90.0.160/svn/webstack/releases/CKB-9521/httpd-webrail/ httpd-webrail cd httpd-webrail/rewrites make clean stage-kanban env=all remoteuser=qastage kanban_branch=features * find new version-tag http://10.90.0.160/svn/webstack/tags/ qatage/qastage * on staging box sudo chown -R distuser:rcom /rt/staging/qa/apache-rewrites/Release-1.0-287 cd /rt/staging/qa/apache-rewrites/Release-1.0-287 tar xzvf apache-rewrites-1.0-287.tgz * deploy apache rewrie export env=qa2 cd /rt/staging/qa/apache-rewrites/Release-1.0-287 make dist env=${env} remoteuser=distuser servers="${env}-web-01 ${env}-web-02" make install env=${env} remoteuser=distuser servers="${env}-web-01 ${env}-web-02" make restart-apache env=${env} remoteuser=distuser servers="${env}-web-01 ${env}-web-02" * detail reference CKB-9521 mdsbulk synchronization 1. In realtime db, check job name mysql> SELECT * FROM `bulk_job_name_value` WHERE `value` = 'quoteview' LIMIT 0,1000; +---------------------+-----------+ | name | value | +---------------------+-----------+ | tables_symbology_V2 | quoteview | +---------------------+-----------+ 1 row in set (0.05 sec) 2. in mdsbulk-01 Stop crond and running job ~$ sudo /etc/init.d/crond stop Run job in mdsbulk-01 sudo /rt/apps/md-jobs/bin/stagejobrequest.sh 'f' symbology_V2 Restart crond once the synchronization is done ~$ sudo /etc/init.d/crond start rcom elastic search docs * https://thehub.thomsonreuters.com/docs/DOC-1069931 - Overview of Elasticsearch in Rcom * https://thehub.thomsonreuters.com/docs/DOC-990922 - Prepare dev evnvironment of Elasticsearch * https://thehub.thomsonreuters.com/docs/DOC-1643546 - How to reindex/catchup data for Elasticsearch * https://thehub.thomsonreuters.com/docs/DOC-1108802 - Elasticsearch Servers in RCOM * https://thehub.thomsonreuters.com/docs/DOC-1069931 - Overview of Elasticsearch in Reuters.com Journalist profile page * https://thehub.thomsonreuters.com/docs/DOC-1646442 - Overview of Journalist profile page on Reuters.com